This invention relates generally to testing methodologies and more particularly to metrics-related testing of an Operational Support System (OSS) of an incumbent provider for compliance with a regulatory scheme.
In the United States, a regulatory entity of a state or other jurisdiction may be tasked with considering whether, and to what extent, an ILEC is in compliance with the requirements of Section 271 of The Telecommunications Act of 1996 (xe2x80x9cthe 1996 Actxe2x80x9d). The 1996 Act, together with associated Federal Communications Commission (FCC) interpretations, generally require an ILEC seeking to enter the market for long distance telecommunications services to: (1) provide the relevant CLEC community with non-discriminatory access to the ILEC""s OSSs, subject to suitable terms and conditions; (2) provide documentation and support necessary for CLECs to use these OSSs; and (3) demonstrate that these OSSs are operationally ready and provide an acceptable level of performance to the CLEC community. Compliance with these requirements should, for example, allow CLECs to obtain pre-ordering information, fulfill orders for resale services and unbundled network elements (UNEs), manage problems, and obtain billing information in a manner that is non-discriminatory as compared to the ILEC""s own retail operations.
The regulatory entity may retain an independent testing entity to test and evaluate the readiness of the ILEC""s OSS interfaces, documentation, and processes to support local market entry by CLECs. Based on its testing activities, the testing entity may generate a report documenting the manner in which its testing activities were conducted, results of the testing activities, and its evaluations based on the results. The audience for the report will generally fall into two main categories: (1) readers who will use the report in connection with the regulatory process, and (2) any other interested parties who may have some stake in the matter and wish to have visibility into the testing activities, results, and evaluations (e.g., the ILEC being evaluated, the CLECs, and other ILECs). While many of the above parties may have some interest in the outcome of the testing activities, typically only the regulatory entity and the ILEC are parties to the contract with the testing entity.
According to the present invention, problems and disadvantages associated with testing OSSs of an incumbent provider may be reduced or eliminated.
In one embodiment of the present invention, a method is provided for testing one or more operational support systems (OSSs) of an incumbent provider for compliance with a regulatory scheme, the method performed by an independent testing entity attempting to emulate a competitive provider that would access the OSSs in attempting to compete with the incumbent provider in an open market. For each test of an OSS, the method includes performing a test entrance review, according to predetermined review guidelines and prior to initiation of active testing of the OSS for the test, to ensure that all required entrance criteria for the test have been satisfied. Active testing of the OSS is conducted for the test according to a written detailed test plan for the test, and performance of the incumbent provider during active testing is evaluated according to predetermined evaluation criteria for the test. A written exception is generated for each aspect of the test for which the testing entity determines during active testing that the incumbent provider fails to satisfy one or more applicable predetermined evaluation criteria, the exception describing such failure and the potential impact of the failure on competitive providers that would access the OSS in attempting to compete with the incumbent provider in an open market. For each exception, the method includes submitting the exception to the incumbent provider for review and receiving a written response to the exception from the incumbent provider, the response describing one or more planned corrective activities of the incumbent provider to remediate the associated failure. Subsequent to the corrective activities being performed, additional active testing of the OSS is conducted according to the detailed test plan with respect to the corresponding aspect of the test, and performance of the incumbent provider during the additional active testing is evaluated according to the evaluation criteria applicable to the corresponding aspect of the test. If the exception is cleared, based on the incumbent provider satisfying the applicable evaluation criteria during the additional active testing, a written closure statement is generated for the exception. If the exception is not cleared, based on the incumbent provider again failing to satisfy the applicable evaluation criteria during the additional active testing, then the submitting, receiving, conducting, and evaluating steps are repeated until the exception is cleared or a predetermined time period for the test has elapsed. The method further includes generating test results for the test; performing a test exit review, according to the predetermined review guidelines and subsequent to completion of active testing, to ensure that active testing was conducted in accordance with the detailed test plan, that the test results are appropriately supported, and that all required exit criteria for the test have been satisfied; and issuing a final report for the test providing a sufficient basis for a regulatory entity administering the regulatory scheme to determine the compliance of the incumbent provider with the regulatory scheme.
In another embodiment of the present invention, a method is provided for testing one or more operational support systems (OSSs) of an incumbent provider for compliance with a regulatory scheme, the method performed by an independent testing entity attempting to emulate a competitive provider that would access the OSSs in attempting to compete with the incumbent provider in an open market. For each test of an OSS, the method includes conducting active testing of the OSS for the test according to a written detailed test plan for the test and evaluating performance of the incumbent provider during active testing according to predetermined evaluation criteria for the test. A written exception is generated for each aspect of the test for which the testing entity determines during active testing that the incumbent provider fails to satisfy one or more applicable evaluation criteria, the exception describing such failure and the potential impact of the failure on competitive providers that would access the OSS in attempting to compete with the incumbent provider in an open market. For each exception, an exception resolution process is conducted and includes recording the exception in a master list of exceptions comprising an exception identifier for each exception and a status of each exception, the status for the exception being updated as appropriate during the exception resolution process. The exception is submitted to the incumbent provider for review, the exception submitted to the incumbent provider being considered a draft exception, the exception having a draft status in the master list of exceptions. A written response to the draft exception is received from the incumbent provider challenging the draft exception on one or more factual bases. It is determined in cooperation with a regulatory entity administering the regulatory scheme that the draft exception should not be withdrawn, the draft exception being considered an open exception in response to the determination that the draft exception should not be withdrawn, the exception having an open status in the master list of exceptions. A written response to the open exception is received from the incumbent provider describing one or more planned corrective activities of the incumbent provider to remediate the associated failure. Subsequent to the corrective activities being performed, additional active testing of the OSS is conducted according to the detailed test plan with respect to the corresponding aspect of the test, and performance of the incumbent provider during the additional active testing is evaluated according to applicable evaluation criteria. If the open exception is cleared, based on the testing entity determining in cooperation with the regulatory entity that the incumbent provider has satisfied applicable evaluation criteria during the additional active testing, then a written closure statement is generated for the open exception, the open exception being considered a closed exception in response to generation of the closure statement, the exception having a closed status in the master list of exceptions. If the open exception is not cleared, based on the testing entity determining in cooperation with the regulatory entity that the incumbent provider has again failed to satisfy applicable evaluation criteria during the additional active testing, then the submitting, receiving, conducting, and evaluating steps are repeated until either the open exception is cleared or a predetermined time period for the test has elapsed. The method further includes generating test results for the test and issuing a final report for the test providing a sufficient basis for the regulatory entity to determine the compliance of the incumbent provider with the regulatory scheme.
In another embodiment of the present invention, a system is provided for use in connection with testing one or more operational support systems (OSSs) of an incumbent provider by an independent testing entity for compliance with a regulatory scheme, the testing entity attempting to emulate a competitive provider that would access the OSSs in attempting to compete with the incumbent provider in an open market. The system operates on one or more computer systems at one or more locations and includes a centralized repository supporting comprehensive cataloging, tracking, and reporting of exceptions across multiple tests, test domains, and jurisdictions, the centralized repository maintaining a master list of exceptions comprising an exception identifier for each exception and a status of each exception, the status for each exception being updated as appropriate during an exception resolution process. At least one exception has been generated for a test in response to the testing entity determining, during active testing of the OSS for the test according to a written detailed test plan for the test, that the incumbent provider failed to satisfy a predetermined evaluation criterion for the test. The exception describes such failure and the potential impact of the failure on competitive providers that would access the OSS in attempting to compete with the incumbent provider in an open market. The exception has been recorded in the master list of exceptions and has been submitted to the incumbent provider for review as a draft exception having a draft status in the master list of exceptions. The exception has been allowed to proceed as an open exception having an open status in the master list of exceptions in response to the testing entity receiving a written response to the draft exception from the incumbent provider challenging the draft exception on one or more factual bases and the testing entity determining in cooperation with a regulatory entity administering the regulatory scheme that the draft exception should not be withdrawn. The exception has been designated as a closed exception having a closed status in the master list of exceptions in response to the testing entity receiving a written response to the open exception from the incumbent provider describing one or more planned corrective activities of the incumbent provider to remediate the associated failure, where the testing entity: has conducted additional active testing of the OSS according to the detailed test plan with respect to the corresponding aspect of the test subsequent to the corrective activities being performed; has evaluated performance of the incumbent provider during the additional active testing according to the evaluation criterion; has cleared the exception based on the testing entity determining in cooperation with the regulatory entity that the incumbent provider has satisfied the evaluation criterion during the additional active testing; and has generated a written closure statement for the exception. The system further includes a web server operable to make the master list of exceptions accessible to a plurality of interested parties in the form of Hypertext Markup Language (HTML) pages communicated to the interested parties from the web server.
Certain embodiments of the present invention may provide one or more technical advantages as described and otherwise indicated in the figures, descriptions, and claims included herein. Advantages may include, in particular embodiments, a comprehensive, effective, efficient, and reliable testing methodology, suitable for a variety of environments, jurisdictions, and particular needs. The present invention may encompass discrete elements of this methodology, in any suitable combination, or may encompass the high level methodology as a whole, as limited only by the appended claims. These technical advantages may be apparent to those skilled in the art from the figures, descriptions, and claims included herein.